Face orientation detection apparatus, face orientation detection method, and computer memory product

ABSTRACT

The CPU of a face orientation detection apparatus which acquires an image frame photographed by a camera through a cable detects the face region in the horizontal direction of the acquired image frame by executing computer programs stored on a hard disk. Moreover, the CPU detects the eye position in the vertical direction from the image frame, and detects the nose position in the horizontal direction based on the detected eye position. Furthermore, the CPU detects the orientation of the face included in the image frame, based on the detected nose position and face region.

BACKGROUND OF THE INVENTION

The present invention relates to a face orientation detection apparatus,a face orientation detection method and a computer memory product, fordetecting the orientation of a face from image data acquired byphotographing the face.

As an apparatus for assisting driving of a vehicle such an automobile,there has been proposed an apparatus for detecting the orientation of adriver's face based on image data obtained by photographing the face ofthe driver with a camera mounted in the vehicle at a position capable ofphotographing the face of the driver. With the use of such an apparatus,it is possible to detect inattentive driving from the orientation of thedriver's face detected by the apparatus, and also possible to build asystem for warning the driver of inattentive driving (see, for example,Japanese Patent Application Laid-Open No. 6-243367/1994).

However, it is often the case that an apparatus as mentioned above isconstructed to determine the orientation of a driver's face by storingimage data acquired by photographing the face of the driver in aplurality of orientations in advance, comparing these image data withimage data sequentially acquired when the vehicle is running, andselecting image data of an orientation having a higher degree ofcorrelation. In such a structure, since a plurality of pieces of imagedata acquired by photographing the face facing respective orientationsneed to be stored in advance, the memory capacity for storing therespective image data is very large. Moreover, since the image data witha high degree of correlation is selected for each piece of sequentiallyacquired image data, it takes a long time to determine the faceorientation.

Furthermore, owing to the process for judging the face orientation basedon the image data acquired by photographing a predetermined driver inadvance, if another driver drives the vehicle, it is impossible to judgethe face orientation. Besides, even when the same driver drives thevehicle, if the distance between the camera mounted in the vehicle andthe driver as an object to be photographed differs from the distancewhen the image data was photographed, there is a difference in the sizeof the face region of the driver between these image data, and it isnecessary to perform an enlargement or reducing process on the faceregion of sequentially acquired image data. Thus, such an apparatussuffers from the problem that it can not flexibly meet variousconditions. Besides, since the apparatus of Japanese Patent ApplicationLaid-Open No. 6-243367 (1994) is constructed to extract the tear ductsof both eyes and both ends of the lips from image data acquired byphotographing the face of a driver and detect the orientation of thedriver's face, based on the positional relationship among the extractedfour points, if sequentially acquired image data does not include thelips of the driver, for example, there arises a problem that theapparatus can not detect the orientation of the driver's face.

BRIEF SUMMARY OF THE INVENTION

The present invention has been made with the aim of solving the aboveproblem, and it is an object of the present invention to provide a faceorientation detection apparatus, a face orientation detection method anda computer memory product, which detect the orientation of a face basedon sum values in the vertical direction of image data acquired byphotographing the face, and are thus capable of reducing the amount ofdata to be processed and shortening the processing time because theorientation of the face is detected by the process based on thecalculated sum values.

Another object of the present invention is to provide a face orientationdetection apparatus which uses a characteristic table storingcharacteristic data of the sum values of pixel data in the verticaldirection of the image data of a plurality of face orientations acquiredwhen the face was photographed in the respective orientations, extractsthe characteristic data of sum values in the vertical direction of imagedata sequentially acquired by photographing the face, and selects anddetermines the face orientation corresponding to the extractedcharacteristic data, and is thus capable of reducing the memory capacityfor storing data for use in the face orientation judging process becauseit is sufficient to store only the characteristic data in advance.

Still another object of the present invention is to provide a faceorientation detection apparatus, a face orientation detection method anda computer memory product, which detect the eye position in the verticaldirection based on sum values in the horizontal direction of image dataacquired by photographing a face and detect the face orientation basedon the sum values in the vertical direction in a vicinity region of thedetected eye position, and are thus capable of reducing the amount ofdata to be processed and shortening the processing time because the faceorientation is detected by the process based on the calculated sumvalues in the vicinity region of eyes for one piece of image data.

Yet another object of the present invention is to provide a faceorientation detection apparatus, a face orientation detection method anda computer memory product, which detect the face region from image dataacquired by photographing a face, detect the nose position in thehorizontal direction from the image data, further detect the faceorientation based on the detected nose position and face region, and arethus capable of accurately calculating not only the face orientation butalso the face angle.

A further object of the present invention is to provide a faceorientation detection apparatus which compares the sum value in thevertical direction of image data acquired by photographing a face and athreshold stored in advance, judges that pixel data calculated into thesum value is within the face region when the sum value is larger thanthe threshold, detects the face region based on this judgment result,and is thus capable of accurately specifying the face region in thehorizontal direction and accurately detecting the face orientation basedon the accurately detected face region and nose position.

A further object of the present invention is to provide a faceorientation detection apparatus which calculates a variation in thehorizontal direction of the sum values in the vertical direction ofimage data acquired by photographing a face, specifies the face outlinein the horizontal direction based on the calculated variation, detectsthe face region based on the specified outline, and is thus capable ofdetecting the face region more accurately based on the specifiedoutline.

A further object of the present invention is to provide a faceorientation detection apparatus which detects the eye position in thevertical direction based on the sum values in the horizontal directionof image data acquired by photographing a face, specifies a localminimum value of the sum values in the vertical direction in a regionlocated lower than the detected eye position by a preset predeterminednumber of pixels in the vertical direction, detects that a position inthe horizontal direction of a sum value specified as the local minimumvalue is the nose position, and is thus capable of detecting the nosebased on the pixel data in a region excluding a glasses frame.

A further object of the present invention is to provide a faceorientation detection apparatus which, when a plurality of sum values inthe vertical direction calculated for a region lower than eyes arespecified as local minimum values, calculates the distance in thehorizontal direction between these sum values, specifies the pixel datacalculated into the sum values specified as the local minimum values aspixel data in the outline portion of the nose if the calculated distanceis shorter than a preset predetermined distance, detects that the centerportion in the horizontal direction of the two sum values separated bythe calculated distance is the nose position, and is thus capable ofaccurately detecting the nose position.

According to a first aspect of the face orientation detection apparatusof the invention, there is provided an apparatus for detecting theorientation of a face from image data acquired by photographing theface, the image data being composed of a plurality of pixel data alignedin horizontal direction and vertical direction, respectively,characterized by comprising: vertical adding means for adding therespective pixel data in the vertical direction of the image data; andorientation detecting means for detecting a face orientation based onsum values calculated by the vertical adding means.

In the first aspect, since the respective pixel data in the verticaldirection of the image data acquired by photographing the face are addedand the face orientation is detected based on a plurality of sum valuescalculated, the face orientation is detected by the process based on thecalculated sum values for one piece of image data, and consequently theamount of data to be processed is reduced, the processing time isshortened, and a speedy detection process is realized for sequentiallyacquired image data. Here, since the face region tends to bephotographed more brightly compared to the hair region and thebackground region, if the face region and hair region, for example, thehair region can be specified based on the sum values obtained by addingthe respective pixel data (for example, brightness values) in thevertical direction of the acquired image data, it is possible to detectthe face orientation such as whether the face faces forward or sideward.

According to a second aspect of the invention, in the first aspect, theface orientation detection apparatus is characterized by comprising:extracting means for extracting characteristic data of sum valuescalculated by the vertical adding means; and a characteristic tablestoring the characteristic data in association with a plurality of faceorientations, wherein the orientation detecting means comprisesselecting means for selecting, from the characteristic table, a faceorientation corresponding to the characteristic data extracted by theextracting means.

In the second aspect, by using the characteristic table that stores thecharacteristic data of sum values of pixel data in the verticaldirection of image data of a plurality of face orientations when theface was photographed in the respective orientations, extracting thecharacteristic data of sum values in the vertical direction of imagedata sequentially acquired by photographing the face, and selecting anddetermining the face orientation corresponding to the extractedcharacteristic data from the characteristic table, it is sufficient tostore only the above-mentioned characteristic data in advance and it ispossible reduce the memory capacity for storing data for use in the faceorientation judging process. Moreover, since the face orientation isdetected by the process based on the characteristic data as describedabove, for example, even if an object to be photographed such as adriver differs or if the photographed image data does not include a partof the face, the face direction can be detected accurately based on thephotographed image data.

According to a third aspect of the face orientation detection apparatusof the invention, there is provided an apparatus for detecting theorientation of a face from image data acquired by photographing theface, characterized by comprising: horizontal adding means for addingrespective pixel data in the horizontal direction of the image data; eyedetecting means for detecting an eye position in the vertical directionbased on sum values calculated by the horizontal adding means; eyevicinity adding means for adding respective pixel data in the verticaldirection in a vicinity region of the eye position detected by the eyedetecting means; and orientation detecting means for detecting a faceorientation based on the sum values calculated by the eye vicinityadding means.

In the third aspect, the respective pixel data in the horizontaldirection of the image data acquired by photographing the face areadded, and the eye position in the vertical direction is detected basedon a plurality of sum values calculated. Moreover, by adding the pixeldata in the vertical direction in the vicinity region of the detectedeye position and detecting the face orientation based on a plurality ofsum values calculated, the face orientation is detected by the processbased on the sum values calculated in the vicinity region of eyes forone piece of image data, and consequently the amount of data to beprocessed is reduced and the processing time is shortened. Here, at theeye position in the vertical direction, the ear region and the hairregion are often present, and the pixel data in the ear and hair regionstends to show a large change, and therefore it is possible specify theear and hair regions based on the sum values calculated by the eyevicinity adding means in such a vicinity region of the eye position. Forexample, if the hair region can be specified, it is possible to detectthe face orientation such as whether the face faces forward or sideward.Consequently, even if an object to be photographed such as a driverdiffers or if the photographed image data does not include a vicinityregion of the mouth and nose, the face orientation can be detectedaccurately based on the image data including the eyes.

According to a fourth aspect of the invention, in the third aspect, theface orientation detection apparatus is characterized by comprising:maximum value specifying means for specifying a maximum value of the sumvalues calculated by the horizontal adding means; local minimum valuespecifying means for specifying a local minimum value of the sum valuescalculated by the horizontal adding means; and selecting means forselecting a sum value which is located higher in the vertical directionthan a sum value specified as the maximum value by the maximum valuespecifying means and is specified as the local minimum value by thelocal minimum value specifying means, wherein the eye detecting meansdetects that a position in the vertical direction of the sum valueselected by the selecting means is the eye position.

In the fourth aspect, the maximum value and local minimum value of aplurality of sum values obtained by adding the pixel data in thehorizontal direction of the image data acquired by photographing theface are specified, and, when a position in the vertical direction of asum value specified by the local minimum value which is located higherin the vertical direction than a sum value specified as the maximumvalue is detected as the eye position, for example, even if an object tobe photographed such as a driver differs, the face orientation isdetected by the process based on the sum values calculated by the eyevicinity calculating means in the vicinity region of eyes based on thespecified eye position, and consequently the amount of data to beprocessed is reduced, the processing time is shortened, and the faceorientation can be detected accurately. The reason for this is that,since the eye region tends to be photographed more darkly compared tothe skin region and the cheek portion tends to be photographed mostbrightly in the face region, the sum value specified as the maximumvalue by the maximum value specifying means as described above is oftenthe sum value of pixel data at the cheek position, and there is a highpossibility that the sum value specified as the local minimum value andlocated higher in the vertical direction than the cheek position is thesum value of pixel data at the eye position.

According to a fifth aspect of the invention, in the third aspect, theface orientation detection apparatus is characterized by comprising:local minimum value specifying means for specifying a local minimumvalue of the sum values calculated by the horizontal adding means; andselecting means for selecting a sum value which is specified as thelocal minimum value by the local minimum value specifying means in thesecond highest position in the vertical direction of the sum valuescalculated by the horizontal adding means, wherein the eye detectingmeans detects that a position in the vertical direction of the sum valueselected by the selecting means is the eye position.

In the fifth aspect, a plurality of local minimum values of sum valuesobtained by adding pixel data in the horizontal direction of the imagedata acquired by photographing the face are specified, and, when aposition in the vertical direction of the second highest sum value ofthe image data among the sum values specified as the local minimumvalues is detected as the eye position, for example, even if an objectto be photographed such as a driver differs or if the photographed imagedata does not include a vicinity region of the mouth and nose, the faceorientation can be detected by the process based on the sum valuescalculated by the eye vicinity calculating means in the vicinity regionof eyes based on the specified eye position. The reason for this isthat, since the local minimum value at the highest position of imagedata among the specified local minimum values is often the sum value ofpixel data at the eyebrow position, and the local minimum value at thesecond highest position is often the sum value of pixel data at the eyeposition.

According to a sixth aspect of the face orientation detection apparatusof the present invention, there is provided an apparatus for detectingthe orientation of a face from image data acquired by photographing theface, characterized by comprising: region detecting means for detectinga face region in the horizontal direction from the image data; nosedetecting means for detecting a nose position in the horizontaldirection from the image data; and orientation detecting means fordetecting a face orientation based on the nose position detected by thenose detecting means and the region detected by the region detectingmeans.

In the sixth aspect, by detecting the face region from the image dataacquired by photographing the face, detecting the nose position in thehorizontal direction from the image data and further detecting the faceorientation based on the detected nose position and face region, it ispossible to accurately detect not only the face orientation, but alsothe face angle.

According to a seventh aspect of the invention, in the sixth aspect, theface orientation detection apparatus is characterized by comprising:threshold storing means for storing thresholds; vertical adding meansfor adding the respective pixel data in the vertical direction of theimage data; comparing means for comparing a sum value calculated by thevertical adding means and a threshold stored in the threshold storingmeans, wherein, when the sum value is larger than the threshold, theregion detecting means judges that the pixel data calculated into thesum value are within the face region.

In the seventh aspect, a sum value in the vertical direction of theimage data acquired by photographing the face and a threshold stored inthe threshold storing means in advance are compared, and, when the sumvalue is larger than the threshold as a result of the comparison, thepixel data calculated into the sum value are judged to be within theface region, and the face region is detected based on this judgmentresult. Since the face region tends to be photographed more brightlycompared to the background region, it is possible to accurately specifythe face region based on the sum values calculated by the verticalcalculating means and accurately detect the face orientation based onthe accurately detected face region and nose position.

According to an eighth aspect of the invention, in the sixth aspect, theface orientation detection apparatus is characterized by comprising:vertical adding means for adding the respective pixel data in thevertical direction of the image data; variation calculating means forcalculating a variation in the horizontal direction of the sum valuescalculated by the vertical adding means; and specifying means forspecifying a face outline in the horizontal direction based on thevariation calculated by the variation calculating means, wherein theregion detecting means detects the face region based on the face outlinespecified by the specifying means.

In the eighth aspect, by calculating a variation in the horizontaldirection of the sum values in the vertical direction of the image dataacquired by photographing the face, specifying the face outline in thehorizontal direction based on the calculated variation and detecting theface region based on the specified face outline, the pixel data atpositions where the variation of the pixel data in the horizontaldirection changes largely, such as a point of change from the backgroundregion to the face region and a point of change from the face region tothe background region, can be specified as the pixel data in the outlineportion of the face in the horizontal direction, and the face region canbe detected more accurately based on the specified outline.

According to a ninth aspect of the invention, in any one of the sixththrough eighth aspects, the face orientation detection apparatus ischaracterized by comprising: pixel number storing means for storing apredetermined number of pixels; horizontal adding means for adding therespective pixel data in the horizontal direction of the image data; eyedetecting means for detecting an eye position in the vertical directionbased on the sum values calculated by the horizontal adding means; undereye adding means for adding the pixel data in the vertical direction ina region located lower than the eye position detected by the eyedetecting means by the number of pixels stored in the pixel numberstoring means; and local minimum value specifying means for specifying alocal minimum value of the sum values calculated by the under eye addingmeans, wherein a position in the horizontal direction of the sum valuespecified as the local minimum value by the local minimum valuespecifying means is detected as a nose position.

In the ninth aspect, the eye position in the vertical direction isdetected based on the sum values in the horizontal direction of theimage data acquired by photographing the face, and sum values of pixeldata in the vertical direction are calculated in a region located lowerthan the detected eye position by the preset predetermined number ofpixels. Moreover, by specifying the local minimum value of thecalculated sum values and detecting that a position in the horizontaldirection of the sum value specified as the local minimum value is thenose position, the nose position can be detected by a simple process.Here, since the outline portion of the nose tends to be photographeddarkly in the face region, the sum value specified as the local minimumvalue is often the image data in the outline portion of the nose, andthe nose position can be detected accurately based on the sum valuespecified as the local minimum value in the above-mentioned manner.Further, by detecting the nose based on the pixel data in the regionlocated lower than the eye position by the predetermined number ofpixels, even if the image data includes a glasses frame, it is possibleto detect the nose based on the pixel data in the region excluding theglasses frame, thereby detecting the nose without being influenced bythe glasses frame.

According to a tenth aspect of the invention, in the ninth aspect, theface orientation detection apparatus is characterized by comprising:distance storing means for storing a predetermined distance; distancecalculating means for, when a plurality of local minimum values arespecified by the local minimum value specifying means, calculating adistance in the horizontal direction between sum values specified as thelocal minimum values; and comparing means for comparing the distancecalculated by the distance calculating means and the predetermineddistance stored in the distance storing means, wherein, when thedistance calculated by the distance calculating means is shorter thanthe predetermined distance, a center portion in the horizontal directionof the two sum values separated by the distance is detected as the noseposition.

In the tenth aspect, when a plurality of sum values are specified localminimum values among sum values in the vertical direction calculated forthe under eye region, the distance in the horizontal direction betweenthese sum values is calculated. When the calculated distance is shorterthan the preset predetermined distance, the pixel data calculated intothe sum values specified as the local minimum values can be specified asthe pixel data in the outline portion of the nose, and the nose positioncan be detected accurately by detecting the center portion in thehorizontal direction of the two sum values separated by the distance asthe nose position.

The above and further objects and features of the invention will morefully be apparent from the following detailed description withaccompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of the configuration of aface orientation detection apparatus of the present invention;

FIG. 2 is a flowchart showing the procedure of a face region detectionprocess performed by the face orientation detection apparatus;

FIGS. 3A through 3D are views for explaining the face region detectionprocess performed by the face orientation detection apparatus;

FIG. 4 is a flowchart showing the procedure of an eye position detectionprocess performed by the face orientation detection apparatus;

FIGS. 5A through 5D are views for explaining the eye position detectionprocess performed by the face orientation detection apparatus;

FIG. 6 is a flowchart showing the procedure of a nose position detectionprocess performed by the face orientation detection apparatus;

FIGS. 7A through 7C are views for explaining the nose position detectionprocess performed by the face orientation detection apparatus;

FIGS. 8A and 8B are views for explaining the nose position detectionprocess performed by the face orientation detection apparatus;

FIG. 9 is a view showing an image frame including a glasses frame;

FIG. 10 is a flowchart showing the procedure of a face orientationdetection process performed by the face orientation detection apparatus;

FIG. 11 is a flowchart showing the procedure of a face orientationdetection process performed by the face orientation detection apparatus;

FIGS. 12A through 12C are views for explaining the face orientationdetection process performed by the face orientation detection apparatus;

FIGS. 13A through 13D are views for explaining the face orientationdetection process performed by the face orientation detection apparatus;and

FIG. 14 is a view for explaining a face angle calculation processperformed by the face orientation detection apparatus.

DETAILED DESCRIPTION OF THE INVENTION

The following description will explain a face orientation detectionapparatus of the present invention in detail, based on the drawingsillustrating an embodiment thereof. The face orientation detectionapparatus of this embodiment is constructed to detect the orientation ofa driver's face, for example, based on an image photographed by a camera(imaging device) mounted in a position capable of photographing the faceof the driver on a vehicle such as an automobile. Note that the cameracan be mounted, for example, near the mount position of a mirror orsteering wheel, and acquires, for example, 30 image frames (image data)per second.

FIG. 1 is a block diagram showing an example of the configuration of theface orientation detection apparatus of the present invention. In FIG.1, numeral 1 represents the face orientation detection apparatus of thepresent invention, and this face orientation detection apparatus 1 isconnected to a camera 2 through a cable C. Although the camera 2 and theface orientation detection apparatus 1 are connected with the cable Cfor exclusive use, it is also possible to connect them through a networksuch as a LAN (Local Area Network) in the vehicle.

The camera 2 controls the operations of the following hardware sectionsconnected through a bus 27 to an MPU (Micro Processor Unit) 20 as acontrol center. Connected to the bus 27 are a ROM 21 storing computerprograms showing the control procedure for the MPU 20; a RAM 22 forstoring temporarily various data generated during control operationperformed by the MPU 20; an imaging section 23 having a CCD (ChargeCoupled Device); an A/D (Analog/Digital) converter 24 for converting ananalog image frame acquired by the imaging section 23 into a digitalimage frame; a frame memory 25 composed of a RAM for storing temporarilythe digital image frame converted by the A/D converter 24; and acommunication interface 26 for performing data transmission through thecable C.

The MPU 20 controls the above-mentioned various hardware sectionsthrough the bus 27 while measuring a predetermined timing by clock means(not shown) installed therein, and sequentially executes variouscomputer programs necessary for the operation of the camera 2, stored inthe ROM 21. Besides, the A/D converter 24 of this embodiment convertsanalog image frames sequentially inputted from the imaging section 23into digital image frames in which each pixel is represented by, forexample, 256 gray levels (1 byte).

The camera 2 with the above-mentioned structure captures an image framewith a predetermined timing using the imaging section 23, and thecaptured image frame is converted into a monochrome digital image frameby the A/D converter 24, temporarily stored in the frame memory 25, andinputted into the face orientation detection apparatus 1 through thecommunication interface 26. Note that it is also possible to use acamera for acquiring color image frames.

The face orientation detection apparatus 1 comprises a CPU (CentralProcessing Unit) 10 as a control center; a hard disk 11 (hereinafterreferred to as HD) storing computer programs showing the controlprocedure for the CPU 10; a RAM 12 for storing temporarily various datagenerated during control operation performed by the CPU 10; a framememory 13 composed of a RAM or the like for storing temporarily an imageframe acquired through the cable C; and a communication interface 14 forperforming data transmission through the cable C.

The CPU 10 comprises clock means (not shown), controls theabove-mentioned various hardware sections through a bus 15 whilemeasuring timing, and sequentially executes computer programs stored inthe HD 11. Moreover, the CPU 10 comprises an eye undetectable flagregister which is turned on when eyes can not be detected by an eyedetection process performed by the CPU 10, and a nose undetectable flagregister which is turned on when a nose can not be detected by a nosedetection process.

The HD 11 stores a region detection process program for detecting theface region of the driver included in the image frame acquired from thecamera 2 through the cable C by the face orientation detection apparatus1; an eye detection process program for detecting the eye position ofthe driver; a nose detection process program for detecting the noseposition of the driver; a face orientation detection process program fordetecting the orientation of the driver's face; and a characteristictable for use in the face orientation detection process. Further, the HD11 stores various thresholds for use in the respective processes, morespecifically, a predetermined brightness value, a predetermined numberof pixels and a predetermined distance, and the HD 11 also functions asthreshold storing means, pixel number storing means and distance storingmeans.

In the face detection apparatus 1 with the above-mentionedconfiguration, the CPU 10 reads the region detection process programstored in the HD 11 into the RAM 12 and sequentially executes theprogram, and thereby detects the face region in the horizontal directionof the driver included in an image frame acquired from the camera 2,based on the image frame. FIG. 2 is a flowchart showing the procedure ofthe face region detection process performed by the face orientationdetection apparatus 1. By executing the region detection process programstored in the HD 11, the CPU 10 operates as vertical adding means foradding respective pixel data aligned next to each other in the verticaldirection of the image frame, and sequentially adds the pixel dataaligned in the vertical direction of the image frame acquired from thecamera 2 (S1).

FIGS. 3A through 3D are views for explaining the face region detectionprocess performed by the face orientation detection apparatus 1. FIG. 3Ashows an example of the image frame acquired from the camera 2. In FIG.3A, X represents the horizontal direction of the image frame, while Yrepresents the vertical direction of the image frame, and the imageframe comprises a plurality of pixel data aligned in the horizontaldirection X and the vertical direction Y, respectively.

By sequentially adding the pixel data aligned in the vertical directionY of the image frame, the CPU 10 calculates a graph as shown in FIG. 3B.In the graph of FIG. 3B, the abscissa shows the horizontal direction X,while the ordinate shows the calculated sum value, more specifically thesum of brightness. Here, since the face region tends to be photographedmore brightly compared to the hair region and the background region, itis possible to specify the face region based on the sum of brightnesscalculated as described above. The CPU 10 of this embodiment uses apredetermined brightness value as a preset threshold in the HD 11, and,when the sum of brightness calculated as described above is larger thanthe predetermined brightness value, the CPU 10 judges that the pixeldata calculated into the sum of brightness is image data within the faceregion, and specifies the face region (S2).

More specifically, the CPU 10 operates as comparing means for comparingthe sum of brightness calculated as described above and thepredetermined brightness value, and a region with larger sum ofbrightness than the predetermined brightness value as a result of thecomparison of the calculated sum of brightness and the predeterminedbrightness value, for example, a region indicated by F1 in FIG. 3C canbe specified as the face region. Note that, F1 a in FIG. 3C indicatesthe left end position of the face region, and F1 b indicates the rightend position of the face region.

Besides, since the pixel data in the horizontal direction of the imageframe tends to change more largely at a change point from the backgroundregion to the face region and a change point from the face region to thebackground region than other points, it is also possible to detect theface region based on the variation of the sum of brightness calculatedas described above. Thus, for the sum of brightness calculated asdescribed above, the CPU 10 operates as variation calculating means forcalculating the variation between a predetermined number of pixels inthe horizontal direction on the left and right sides of the sum ofbrightness of each position in the horizontal direction X, and operatesas specifying means for specifying a position where there is a largechange in the variation respectively calculated, for example, a positionindicated by F2 a in FIG. 3D, as an outline portion of the face in thehorizontal direction X and specifies the face region with this positionas the left end position of the face region (S3).

Note that, in FIG. 3D, the right end position of the face region is notspecified by a process of specifying the face region based on thevariation. In such a case, as shown in FIG. 3C, the face region isspecified by determining that the right end position Flb specified basedon the predetermined brightness value is the right end position of theface region, and the CPU 10 stores the specified face region in the RAM12 (S4) and terminates the face region detection process.

Next, by reading the eye detection process program stored in the HD 11into the RAM 12 and sequentially executing the program, the CPU 10detects the eye position in the perpendicular direction of the driverincluded in the image frame acquired from the camera 2. FIG. 4 is aflowchart showing the procedure of the eye position detection processperformed by the face orientation detection apparatus 1. By executingthe eye detection process program stored in the HD 11, the CPU 10operates as horizontal adding means for adding pixel data aligned nextto each other in the horizontal direction of the image frame, andsequentially adds the pixel data aligned in the horizontal direction ofthe image frame acquired from the camera 2 (S11).

FIGS. 5A through 5D are views for explaining the eye position detectionprocess performed by the face orientation detection apparatus 1, and,similarly to FIG. 3A, FIG. 5A shows an example of the image frameacquired from the camera 2. By sequentially adding the pixel dataaligned in the horizontal direction X of the image frame, the CPU 10calculates a graph as shown in FIG. 5B. In the graph of FIG. 5B, theordinate shows the vertical direction Y, and the abscissa shows thecalculated sum of brightness.

The CPU 10 also operates as maximum value specifying means forspecifying the maximum value of the sum of brightness calculated asdescribed above, and specifies the maximum value (S12). Here, in theface region, since the cheek portion tends to be photographed mostbrightly, the sum of brightness specified as the maximum value can bejudged to be the sum value of the pixel data in the cheek position, anda position in the vertical direction Y indicated by Max in the graph ofFIG. 5C can be specified as the cheek position as mentioned above.

The CPU 10 also operates as local minimum value specifying means forspecifying the local minimum value of the sum of brightness calculatedas described above, and operates as selecting means for selecting thesum of brightness to be the local minimum value at a position higher inthe vertical direction Y than the sum of brightness specified as themaximum value in step S12 and thereby has the structure of selecting thesum of brightness to be the local minimum value at a position higher inthe vertical direction Y than the sum of brightness specified as themaximum value, and judges whether or not such sum of brightness isselected by itself (S13). Since the eye region tends to be photographedmore darkly compared to the skin region, there is a high possibilitythat the sum of brightness which is located higher in the verticaldirection Y than the cheek position indicated by the position of the sumof the brightness as the maximum value and is specified as the localminimum value is the sum value of the pixel data in the eye position,and therefore a position selected in such a manner and indicated by Minin the graph of FIG. 5C can be a candidate for the eye position.

As described above, when the sum of brightness to be the local minimumvalue at a position higher in the vertical direction Y than the sum ofthe brightness as the maximum value is selected (S13: YES), the CPU 10sequentially detects the sums of brightness to be local minimum valuesfrom the higher position in the vertical direction Y in the image frame,selects the sum of brightness to be a local minimum value in the secondhighest position, and judges whether or not such sum of brightness isdetected (S14). Similarly to the eye region, since the eyebrow regiontends to be photographed more darkly compared to the skin region, aposition Min₁ in FIG. 5D indicates the position of the local minimumvalue at the highest position in the vertical direction Y of the imagedata, and this local minimum value can be judged to be the sum value ofthe pixel data in the eyebrow position, while the position of the localminimum value at the second highest position can be judged to be the sumvalue of the pixel data in the eye position, and thus a position Min₂ ofthe local minimum value at the second highest position can be candidatefor the eye position.

When the local minimum value of the sum of brightness at the secondhighest position in the vertical direction Y is detected (S14: YES), theCPU 10 compares the positions of the local minimum values selected insteps S13 and S14 as mentioned above, and judges whether or not thepositions Min and Min₂ of the two local minimum values are substantiallyidentical (S15). If these positions are substantially identical (S15:YES), the CPU 10 specifies the position of the selected local minimumvalue as the eye position and stores it in the RAM 12 (S16). On theother hand, when a corresponding local minimum value is not selected instep S13 or S14 (S13: NO, S14: NO), and when the positions Min and Min₂of the selected two local minimum values are not substantially identicalin step S15 (S15: NO), the CPU 10 turns on the eye undetectable flagregister stored therein (S17), and terminates the eye detection process.

Next, by reading the nose detection process program stored in the HD 11into the RAM 12 and sequentially executing the program, the CPU 10detects the nose position in the horizontal direction of the driverincluded in the image frame acquired from the camera 2. Note that theCPU 10 executes this nose detection process program only when the eyeposition is detected by the execution of the above-mentioned eyedetection process program.

FIG. 6 is a flowchart showing the procedure of the nose positiondetection process performed by the face orientation detection apparatus1. FIGS. 7A through 7C are views for explaining the nose positiondetection process performed by the face orientation detection apparatus1. By executing the nose detection process program stored in the HD 11,the CPU 10 operates as under eye adding means for adding the pixel dataaligned next to each other in the vertical direction in a region R whichis lower in the vertical direction Y by a predetermined number of pixelspre-stored in the HD 11 than an eye position E detected by theabove-mentioned eye position detection process, and sequentially addsthe pixel data aligned in the vertical direction in this region R undereyes (S21) and calculates a graph as shown in FIG. 7B. Similarly to FIG.3A, FIG. 7A shows an example of the image frame acquired from the camera2. In the graph of FIG. 7B, the abscissa shows the horizontal directionX, and the ordinate shows the calculated sum of brightness.

Next, the CPU 10 specifies the local minimum value of the sum ofbrightness calculated as described above (S22). Here, since the outlineportion of the nose tends to be photographed darkly in the face region,the sum of brightness specified as the local minimum value in thismanner can be specified as the sum value of image data in the outlineportion of the nose, and the positions of local minimum values indicatedby Min₃ and Min₄ in the horizontal direction X in the graph of FIG. 7Bcan be specified as the left and right outline positions of nose.

FIG. 7C is an enlarged view of the vicinity of the position Min₃ in thehorizontal direction X of FIG. 7B. As shown in FIG. 7C, the sum ofbrightness in the focused position Min₃ in the horizontal direction X isthe local minimum value of the sum of brightness which is the smallestvalue between a predetermine number of pixels in the left and right ofthe horizontal direction X. The CPU 10 judges whether or not there are aplurality of local minimum values specified as described above (S23).When there are a plurality of local minimum values (S23: YES), the CPU10 operates as distance calculating means for calculating the distancebetween the positions of two local minimum values in the horizontaldirection X and calculates the distance (S24), and then judges whetheror not the calculated distance is shorter than a predetermined distancepre-stored in the HD 11 (S25).

The above-calculated distance in the horizontal direction X between thepositions Min₃ and Min₄ of local minimum values specified as the outlinepositions of nose indicates the width of nose, and, when this width islonger than the predetermined distance (S25: NO), the CPU 10 judges thatthe nose position can not be detected and turns on the nose undetectableflag register installed therein (S26), and terminates the nose detectionprocess.

On the other hand, when the distance in the horizontal direction Xbetween the two local minimum values is shorter than the predetermineddistance (S25: YES), the CPU 10 judges that the local minimum valuesMin₃ and Min₄ separated by this distance are the sum values of imagedata in the outline positions of nose and calculates the center positionbetween these local minimum values Min₃ and Min₄ (S27), and thenspecifies the calculated center position as the nose position and storesit in the RAM 12 (S28). On the other hand, in step S23, when there isnot a plurality of specified local minimum values (S23: NO), i.e., whenthe left and right outline portions of nose are not accurately includedin the image frame, the CPU 10 specifies the position in the horizontaldirection X of the sum of brightness which is specified as the localminimum value in step S22 as the nose position and stores it in the RAM12 (S29).

FIGS. 8A and 8B are views for explaining the nose position detectionprocess performed by the face orientation detection apparatus 1 when theoutline portions of nose are not accurately photographed. As shown inFIG. 8A, there is a case where the left and right outline portions ofnose can not be accurately photographed due to the face orientation of adriver to be photographed. In such a case, a position Min₅ in thehorizontal direction X of the sum of brightness that is one localminimum value specified in the above-described step S22 is specified asthe nose position.

FIG. 9 is a view showing an image frame including a glasses frame. Asdescribed above, by detecting the nose position based on the image datain the region R located lower than the detected eye position by thepredetermined number of pixels as shown in FIG. 9, for example, even inthe image frame including a glasses frame, the nose position can bedetected without being influenced by the glasses frame, based on thepixel data in the region R excluding the glasses frame.

The following description will explain how the face orientationdetection apparatus 1 for performing the above-described face regiondetection process, eye detection process, and nose detection process toperform a detection process for detecting the face orientation of thedriver of a vehicle from an image frame acquired from the camera 2.FIGS. 10 and 11 show a flowchart of the procedure of the faceorientation detection process performed by the face orientationdetection apparatus 1 of the present invention.

In the face orientation detection apparatus 1, the CPU 10 judges whetheror not an image frame has been acquired from the camera 2 (S31). When animage frame has not been acquired from the camera 2 (S31: NO), the CPU10 waits until an image frame has been acquired, and, when an imageframe has been acquired (S31: YES), the CPU 10 operates as regiondetecting means by executing the region detection process program in theHD 11, and executes the face region detection process explained by theflowchart of FIG. 2 for the acquired image frame (S32) and specifies theface region of the driver included in the image frame.

Next, the CPU 10 operates as eye detecting means by executing the eyedetection process program in the HD 11, and executes the eye detectionprocess explained by the flowchart of FIG. 4 for the acquired imageframe (S33) and specifies the eye position of the driver included in theimage frame. Note that in the eye detection process of this embodiment,as described above, since there is a case where the eye position can notbe detected due to an image frame captured by the camera 2, the CPU 10judges whether or not the eye position can be detected by theabove-described eye detection process, based on whether or not the eyeundetectable flag register installed therein is turned off (S34).

FIGS. 12A through 12C are views for explaining the face orientationdetection process performed by the face orientation detection apparatus1, for an image frame from which the eye position can not be detected.Similarly to FIG. 3A, FIG. 12A shows an example of the image frameacquired from the camera 2. One example of the image frame from whichthe eye position can not be detected is an image frame in which thelocal minimum value of the sum of brightness, which is the sum value ofpixel data aligned in the horizontal direction X of the image framecalculated in the above-described eye detection process, can not bespecified.

Thus, when the eye position can not be detected (S34: NO), the CPU 10operates as extracting means for extracting characteristic data of thesum of brightness that is the sum value of pixel data aligned in thevertical direction Y of the image frame calculated by theabove-described face region detection process in step S32 (S35) Here,FIG. 12C shows a graph of the sum of brightness of pixel data aligned inthe vertical direction Y of the image frame, the characteristic dataincludes data indicating the characteristic that each sum of brightnessincreases moderately as shown in a range of the left end to position X₁in the horizontal direction X, and the characteristic table pre-storedin the HD 11 stores data indicating characteristics as mentioned aboveextracted from the image frames photographed in directions correspondingto the respective orientations of the face.

More specifically, in FIG. 12C, in the horizontal direction X asdescribed above, each sum of brightness increases moderately, and thereis a small change in the sum of brightness. There is often the case thatsuch an image frame represents the region of long hair, and suchcharacteristic data is stored in the characteristic table in associationwith the orientation facing right, and the CPU 10 operates asorientation detecting means based on the extracted characteristic dataand detects the face orientation from the characteristic table in the HD11 (S36). In FIG. 12C, it is specified that the face faces right. Notethat such characteristic data stored in the characteristic table can bestored in the HD 11 in advance, but the user can also set to add thecharacteristic data to the RAM 12.

On the other hand, in step S34, when the eye position is detected (S34:YES), the CPU 10 executes the nose detection process explained by theflowchart of FIG. 6 (S37), and specifies the nose position of the driverincluded in the image frame. Similarly to the eye detection process, inthe nose detection process, since there is a case that the nose positioncan not be detected due to an image frame captured by the camera 2, theCPU 10 judges whether or not the nose position has been detected by theabove-described nose detection process, based on whether or not the noseundetectable flag register installed therein is turned off (S38).

FIGS. 13A through 13D are views for explaining the face orientationdetection process performed by the face orientation detection apparatus1, for an image frame from which the nose position can not be detected.Similarly to FIG. 3A, FIG. 13A shows an example of the image frameacquired from the camera 2. As shown in FIG. 13B, one example of theimage frame from which the nose position can not be detected is an imageframe in which the local minimum value of the sum of brightness, whichis the sum value of pixel data aligned in the vertical direction Y inthe region R lower than the eye position by the predetermined number ofpixels, calculated in the above-described nose detection process can notbe specified.

Thus, when the nose position can not be detected (S38: NO), the CPU 10operates as eye vicinity adding means for calculating the sum ofbrightness as the sum value of pixel data aligned in the verticaldirection Y in a vicinity region of eyes indicated by ER in FIG. 13C,based on the eye position calculated by the eye detection process instep S33, and sequentially adds the pixel data aligned in the verticaldirection Y in the eye vicinity region ER (S39). FIG. 13D shows a graphof the sum of brightness of pixel data aligned in the vertical directionY in the region ER. Similarly to step S35, the CPU 10 extractscharacteristic data of the sum of brightness thus calculated (S40), anddetects the face orientation from the characteristic table in the HD 11,based on the extracted characteristic data (S41).

Note that the characteristic data mentioned here is data showing thecharacteristic that each sum of brightness varies extremely as shown inthe range from the left end to position X₂ in the horizontal directionX. There is often the case that such an image frame represents theregion of short hair, and such characteristic data is stored in thecharacteristic table in association with the orientation facing right,and consequently the CPU 10 can detect that the face of the driver facesright in step S41.

On the other hand, in step S38, when the nose position is detected (S38:YES), the CPU 10 operates as the orientation detecting means byexecuting the face orientation detection process in the HD 11, anddetects the orientation including the face angle, based on the faceregion detected in step S32 and the nose position detected in step S37(S42). FIG. 14 is a view for explaining a face angle calculationprocess. Based on an image frame as shown in FIG. 14, when a face regionF and a nose ridge position N of the driver included in the image frameare detected, the CPU 10 calculates the face angle based, for example,on equation (1) shown below.θ=sin⁻¹ {(n ₁ +n ₂)/2−n ₂}/R  (1)

Here, R is a preset average radius of the head of human being, n₁ is thedistance from the left end position to the nose ridge position in theface region, and n₂ is the distance from the nose ridge position to theright end position in the face region. Here, since R=(n₁+n₂)/2 issatisfied, the face angle θ is obtained as following equation (2).θ=sin⁻¹ {(n ₁ −n ₂)/(n ₁ +n ₂)}  (2)

As described above, based on the image frame photographed by the camera2, the face orientation detection apparatus 1 detects the face region ofthe driver included in the image frame, and detects the eye position ofthe driver based on the image frame. Here, if the eye position can notbe detected, the face orientation is detected based on the sum value ofimage data in the vertical direction Y of the image frame, which iscalculated in detecting the face region. On the other hand, if the eyeposition is detected, the nose position is detected based on thedetected eye position, and the face orientation is detected based on thedetected nose position and the face region detected beforehand. Further,if the nose position can not be detected, the face orientation isdetected based on the image data in a region in the vicinity of the eyeposition detected beforehand. Consequently, even in an image frame fromwhich the eye or nose position can not be detected, it is possible todetect the face orientation more accurately, and, if the nose positioncan be detected, it is possible to detect not only the face orientation,but also the face angle.

For the above-described face orientation detection apparatus 1, byproviding further structure to judge whether the face orientationdetected in the above-mentioned manner is appropriate or not based onthe running direction of the vehicle driven by the driver, it ispossible to judge inattentive driving and warn the driver of inattentivedriving when the face orientation of the driver is not appropriate.Moreover, the above-mentioned face orientation detection process can beapplied not only to an image frame acquired by photographing the driverof a vehicle, but also to image frames acquired by photographing faces.

Note that in the above-described embodiment, although the faceorientation detection apparatus 1 stores computer programs for operatingvarious operations in the HD 11 and realizes the respective operationswhen the CPU 10 executes these computer programs, it is also possible toconstruct the respective operations by hardware.

As this invention may be embodied in several forms without departingfrom the spirit of essential characteristics thereof, the presentembodiment is therefore illustrative and not restrictive, since thescope of the invention is defined by the appended claims rather than bythe description preceding them, and all changes that fall within metesand bounds of the claims, or equivalence of such metes and boundsthereof are therefore intended to be embraced by the claims.

1. An apparatus for detecting orientation of a face from image dataacquired by photographing the face, the image data being composed of aplurality of pixel data aligned in horizontal direction and verticaldirection, respectively, comprising: a horizontal adding section foradding the respective pixel data in the horizontal direction of theimage data; an eye detecting section for detecting an eye position inthe vertical direction based on a plurality of sum values calculated bysaid horizontal adding section; an eye vicinity adding section foradding the respective pixel data in the vertical direction in a vicinityregion of the eye position detected by said eye detecting section; andan orientation detecting section for detecting a face orientation basedon a plurality of sum values calculated by said eye vicinity addingsection.
 2. The face orientation detection apparatus of claim 1, furthercomprising: a vertical adding section for deriving a graph showingvalues of summation of the pixel data, which are calculated by addingthe respective pixel data in the vertical direction of the image data,along the horizontal direction; an extracting section for extractingcharacteristic data of a plurality of sum values calculated by saidvertical adding section; and a characteristic table storing thecharacteristic data in association with a plurality of faceorientations, wherein said orientation detecting section selects, fromsaid characteristic table, a face orientation corresponding to thecharacteristic data extracted by said extracting section.
 3. Anapparatus for detecting orientation of a face from image data acquiredby photographing the face, the image data being composed of a pluralityof pixel data aligned in horizontal direction and vertical direction,respectively, comprising: a horizontal adding section for adding therespective pixel data in the horizontal direction of the image data; aneye detecting section for detecting an eye position in the verticaldirection based on a plurality of sum values calculated by saidhorizontal adding section; an eye vicinity adding section for adding therespective pixel data in the vertical direction in a vicinity region ofthe eye position detected by said eye detecting section; an orientationdetecting section for detecting a face orientation based on a pluralityof sum values calculated by said eye vicinity adding section; a maximumvalue specifying section for specifying a maximum value of the sumvalues calculated by said horizontal adding section; a local minimumvalue specifying section for specifying a local minimum value of the sumvalues calculated by said horizontal adding section; and a selectingsection for selecting a sum value which is located higher in thevertical direction than a sum value specified as the maximum value bysaid maximum value specifying section and is specified as the localminimum value by said local minimum value specifying section, said eyedetecting section detecting that a position in the vertical direction ofthe sum value selected by said selecting section is the eye position. 4.An apparatus for detecting orientation of a face from image dataacquired by photographing the face, the image data being composed of aplurality of pixel data aligned in horizontal direction and verticaldirection, respectively, comprising: a horizontal adding section foradding the respective pixel data in the horizontal direction of theimage data; an eye detecting section for detecting an eye position inthe vertical direction based on a plurality of sum values calculated bysaid horizontal adding section; an eye vicinity adding section foradding the respective pixel data in the vertical direction in a vicinityregion of the eye position detected by said eye detecting section; anorientation detecting section for detecting a face orientation based ona plurality of sum values calculated by said eye vicinity addingsection; a local minimum value specifying section for specifying a localminimum value of the sum values calculated by said horizontal addingsection; and a selecting section for selecting a sum value which isspecified as the local minimum value by said local minimum valuespecifying section in second highest position in the vertical directionof the sum values calculated by said horizontal adding section, said eyedetecting section detecting that a position in the vertical direction ofthe sum value selected by said selecting section is the eye position. 5.An apparatus for detecting orientation of a face from image dataacquired by photographing the face, the image data being composed of aplurality of pixel data aligned in horizontal direction and verticaldirection, respectively, comprising: a region detecting section fordetecting a face region in the horizontal direction from the image data;a nose detecting section for detecting a nose position in the horizontaldirection from the image data; an orientation detecting section fordetecting a face orientation based on the nose position detected by saidnose detecting section and the region detected by said region detectingsection a threshold storing section for storing thresholds; a verticaladding section for adding the respective pixel data in the verticaldirection of the image data; a comparing section for comparing a sumvalue calculated by said vertical adding section and a threshold storedin said threshold storing section, and when the sum value is larger thanthe threshold as a result of comparison in said comparing section, saidregion detecting section judges that the pixel data calculated into thesum value are within the face region.
 6. An apparatus for detectingorientation of a face from image data acquired by photographing theface, the image data being composed of a plurality of pixel data alignedin horizontal direction and vertical direction, respectively,comprising: a region detecting section for detecting a face region inthe horizontal direction from the image data; a nose detecting sectionfor detecting a nose position in the horizontal direction from the imagedata; an orientation detecting section for detecting a face orientationbased on the nose position detected by said nose detecting section andthe region detected by said region detecting section; a vertical addingsection for adding the respective pixel data in the vertical directionof the image data; a variation calculating section for calculating avariation in the horizontal direction of the sum values calculated bysaid vertical adding section; and a specifying section for specifying aface outline in the horizontal direction based on the variationcalculated by said variation calculating section, said region detectingsection detecting the face region based on the face outline specified bysaid specifying section.
 7. An apparatus for detecting orientation of aface from image data acquired by photographing the face, the image databeing composed of a plurality of pixel data aligned in horizontaldirection and vertical direction, respectively, comprising: a regiondetecting section for detecting a face region in the horizontaldirection from the image data; a nose detecting section for detecting anose position in the horizontal direction from the image data; anorientation detecting section for detecting a face orientation based onthe nose position detected by said nose detecting section and the regiondetected by said region detecting section; a pixel number storingsection for storing a predetermined number of pixels; a horizontaladding section for adding the respective pixel data in the horizontaldirection of the image data; an eye detecting section for detecting aneye position in the vertical direction based on a plurality of sumvalues calculated by said horizontal adding section; an under eye addingsection for adding the pixel data in the vertical direction in a regionlocated lower than the eye position detected by said eye detectingsection by the number of pixels stored in said pixel number storingsection; and a local minimum value specifying section for specifying alocal minimum value of the sum values calculated by said under eyeadding section, said nose detecting section detecting that a position inthe horizontal direction of the sum value specified as the local minimumvalue by said local minimum value specifying section is the noseposition.
 8. The face orientation detection apparatus of claim 5,further comprising: a pixel number storing section for storing apredetermined number of pixels; a horizontal adding section for addingthe respective pixel data in the horizontal direction of the image data;an eye detecting section for detecting an eye position in the verticaldirection based on a plurality of sum values calculated by saidhorizontal adding section; an under eye adding section for adding thepixel data in the vertical direction in a region located lower than theeye position detected by said eye detecting section by the number ofpixels stored in said pixel number storing section; and a local minimumvalue specifying section for specifying a local minimum value of the sumvalues calculated by said under eye adding section, wherein said nosedetecting section detects that a position in the horizontal direction ofthe sum value specified as the local minimum value by said local minimumvalue specifying section is the nose position.
 9. The face orientationdetection apparatus of claim 6, further comprising: a pixel numberstoring section for storing a predetermined number of pixels; ahorizontal adding section for adding the respective pixel data in thehorizontal direction of the image data; an eye detecting section fordetecting an eye position in the vertical direction based on a pluralityof sum values calculated by said horizontal adding section; an under eyeadding section for adding the pixel data in the vertical direction in aregion located lower than the eye position detected by said eyedetecting section by the number of pixels stored in said pixel numberstoring section; and a local minimum value specifying section forspecifying a local minimum value of the sum values calculated by saidunder eye adding section, wherein said nose detecting section detectsthat a position in the horizontal direction of the sum value specifiedas the local minimum value by said local minimum value specifyingsection is the nose position.
 10. The face orientation detectionapparatus of claim 7, further comprising: a distance storing section forstoring a predetermined distance; a distance calculating section for,when a plurality of local minimum values are specified by said localminimum value specifying section, calculating a distance in thehorizontal direction between sum values specified as the local minimumvalues; and a comparing section for comparing the distance calculated bysaid distance calculating section and the predetermined distance storedin said distance storing section, wherein, when the distance calculatedby said distance calculating section is shorter than the predetermineddistance as a result of comparison in said comparing section, said nosedetecting section detects that a center portion in the horizontaldirection of the two sum values separated by the distance is the noseposition.
 11. The face orientation detection apparatus of claim 8,further comprising: a distance storing section for storing apredetermined distance; a distance calculating section for, when aplurality of local minimum values are specified by said local minimumvalue specifying section, calculating a distance in the horizontaldirection between sum values specified as the local minimum values; anda comparing section for comparing the distance calculated by saiddistance calculating section and the predetermined distance stored insaid distance storing section, wherein, when the distance calculated bysaid distance calculating section is shorter than the predetermineddistance as a result of comparison in said comparing section, said nosedetecting section detects that a center portion in the horizontaldirection of the two sum values separated by the distance is the noseposition.
 12. The face orientation detection apparatus of claim 9,further comprising: a distance storing section for storing apredetermined distance; a distance calculating section for, when aplurality of local minimum values are specified by said local minimumvalue specifying section, calculating a distance in the horizontaldirection between sum values specified as the local minimum values; anda comparing section for comparing the distance calculated by saiddistance calculating section and the predetermined distance stored insaid distance storing section, wherein, when the distance calculated bysaid distance calculating section is shorter than the predetermineddistance as a result of comparison in said comparing section, said nosedetecting section detects that a center portion in the horizontaldirection of the two sum values separated by the distance is the noseposition.
 13. A method for detecting orientation of a face from imagedata acquired by photographing the face, the image data being composedof a plurality of pixel data aligned in horizontal direction andvertical direction, respectively, comprising: adding using a processor,the respective pixel data in the horizontal direction of the image data;detecting, using a processor, an eye position in the vertical directionbased on a plurality of sum values calculated; adding, using aprocessor, the respective pixel data in the vertical direction in avicinity region of the detected eye position; and detecting, using aprocessor, a face orientation based on a plurality of sum valuescalculated for the vicinity region of eyes.
 14. A computer readablemedium encoded with a computer program that when executed by a computercauses the computer to detect orientation of a face from image dataacquired by photographing the face, the image data being composed of aplurality of pixel data aligned in horizontal direction and verticaldirection, respectively, by performing a method comprising: causing thecomputer to add the respective pixel data in the horizontal direction ofthe image data; causing the computer to detect an eye position in thevertical direction based on a plurality of sum values calculated;causing the computer to add the respective pixel data in the verticaldirection in a vicinity region of the detected eye position; and causingthe computer to detect a face orientation based on a plurality of sumvalues calculated for the vicinity region of eyes.